Skip to content

fix flaky test_audio: skip a settle window after signal onset - #1293

Open
1egoman wants to merge 2 commits into
mainfrom
fix/flaky-test-audio-signal-onset
Open

fix flaky test_audio: skip a settle window after signal onset#1293
1egoman wants to merge 2 commits into
mainfrom
fix/flaky-test-audio-signal-onset

Conversation

@1egoman

@1egoman 1egoman commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Problem

test_audio fails intermittently in CI (macOS and Windows) with e.g.:

Detected sine frequency not within range for channel 0: 38Hz

Every occurrence is the 24000Hz,2ch -> 24000Hz,1ch case reading ~32–38Hz for a 60Hz tone. First seen 2026-07-09.

Root cause (a real early-audio regression, not just a test artifact)

A CI bisect (6 mac reruns per condition, extracting test_audio's own verdict) showed:

Condition test_audio
main baseline (server 1.13.4, single-PC) 3 flake / 6
server pinned 1.13.2 (single-PC) 0 / 6
dual-PC (server 1.13.4) 0 / 6

Both conditions are required, so it's an interaction:

  1. A single-PC subscriber bundles audio SRTP and the (bundle-only, port-0) SCTP datachannel onto one DTLS transport.
  2. livekit-server 1.13.3 bumped pion/webrtc v4.2.15→v4.2.16, whose only media-affecting change is ec1dd730 "Fix bundle-only data channel startup" — it now runs startSCTP for a bundle-only/port-0 datachannel (previously skipped).
  3. startSCTP performs the SCTP association handshake over the shared DTLS transport at setRemoteDescription, contending with the first audio packets → NetEq emits ~300–400ms of concealment at stream start.
  4. The zero-crossing estimate over the first second then reads low and trips the ±20Hz assertion.

Dual-PC is immune (separate subscriber transport); server ≤1.13.2 is immune (never started SCTP for the bundle-only datachannel). The underlying transport contention is a server/pion concern and is being raised separately.

This change (test-side mitigation)

Detect signal onset, then discard a 500ms settle window before measuring 100 frames of steady-state audio. This keeps the estimate off the noisy startup region. (An earlier version of this PR only skipped leading silence, which was insufficient — it left the interleaved startup concealment inside the window and still flaked 1/6.)

🤖 Generated with Claude Code

The zero-crossing frequency estimate counted the first 100 frames the
subscriber stream delivered, including any silence emitted between
subscribing and the arrival of the publisher's first decodable packet.
Enough leading silence drags the estimate below the 40Hz floor
(observed as 32.5-38.5Hz CI failures in the 24kHz publisher case).

Skip frames until one reaches 10% of full scale so the analysis window
only covers actual signal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@1egoman
1egoman requested a review from ladvoc as a code owner July 29, 2026 15:44
@1egoman 1egoman added the internal to tag changes that don't require changelog documentation label Jul 29, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@1egoman

This comment was marked as outdated.

@1egoman 1egoman closed this Jul 29, 2026
The previous onset gate only skipped leading silence; it left the
concealment/gaps that a single-PC subscriber receives at stream start
(while the bundled SCTP association is set up on the shared transport
against a 1.13.3+ server) inside the analysis window, so the flake
persisted (1/6 in CI).

Detect signal onset, then discard a 500ms settle window before measuring
100 frames of steady-state audio. Keeps the zero-crossing estimate off
the noisy startup region.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@1egoman 1egoman reopened this Jul 29, 2026
@1egoman
1egoman marked this pull request as ready for review July 29, 2026 18:45
@1egoman 1egoman changed the title fix flaky test_audio: start frequency analysis at signal onset fix flaky test_audio: skip a settle window after signal onset Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal to tag changes that don't require changelog documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants